• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Method and apparatus for using standard-based mv derivation (pmvd), bidirectional optical flow (bio) or decoder-side mv refinement (dmvr) to refine motion to a bidirectional predicted block are disclosed according to a method of the In the present invention, a first and a second motion compensated reference block associated with the current block in a first and second reference figure from the reference figure list 0 and list 1 are determined respectively. the pmvd process, bio process or dmvr process is applied to generate motion refinement for the current block using reference data corresponding to the first motion compensated reference block and the second motion compensated reference block with no other reference data outside the first one. motion compensated reference block and the second motion compensated reference block so as to reduce the required system memory bandwidth. An adaptive block size method for the block-based bio is also disclosed.
    公开号:BR112019012582A2
    申请号:R112019012582-5
    申请日:2017-12-19
    公开日:2019-11-19
    发明作者:Chen Ching-Yeh；Chuang Tzu-Der；Huang Yu-Wen
    申请人:Mediatek Inc；
    IPC主号:
    专利说明:

    MOVEMENT REFINING METHOD AND APPARATUS FOR VIDEO CODING CROSS-REFERENCE FOR RELATED APPLICATIONS [001] The present invention claims priority of US Provisional Patent Application, Serial No. 62 / 437,759, filed on December 22, 2016 and Application for US Provisional Patent, Serial No. 62 / 439,200, filed on December 27, 2016. Provisional US Patent Applications are hereby incorporated by reference in their entirety.
    FIELD OF THE INVENTION [002] The present invention relates to motion compensation using Standard-based MV Derivation (PMVD), bidirectional optical flow (BIO) or MV Refinement on the Decoder side (DMVR) to refine motion to a block bidirectional. In particular, the present invention relates to the reduction of bandwidth associated with the PMVD, BIO or DMVR process.
    BACKGROUND AND RELATED ART
    Bidirectional Optical Flow (BIO) [003] Bidirectional Optical Flow (BIO) is the motion estimation / compensation technique described in JCTVCC204 (E. Alshina, et al., Bi-directional optical flow,
    Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 3rd Meeting: Guangzhou, CN, 7-15 October 2010, Document: JCTVCC204 ) and VCEG-AZ05 (E. Alshina, et al., Known tools performance investigation for next generation video coding, ITU-T SG 16 Question 6, Video Coding Experts Group (VCEG), 52nd Meeting: 19-26 June 2015 , Warsaw, Poland, Document: VCEG-AZ05). Ο ΒΙΟ derived the refinement of
    Petition 870190056345, of 06/18/2019, p. 32/90
    2/33 sample-level movement based on the assumptions of optical flow and steady movement as shown in Figure 1, where a current pixel 122 in a B-slice (bipredition slice) 120 is predicted by a pixel in reference figure 0 and a pixel in reference figure 1. As shown in Figure 1, the current pixel 122 is predicted by pixel B (112) in reference figure 1 (110) and pixel A (132) in reference figure 0 (130). In Figure 1, v x and v y are pixel displacement vectors in the x direction and y direction, which are derived using a bidirectional optical flow method (BIO). It is applied only for predicted blocks bidirectional, which is predicted from two reference frames corresponding to the previous frame and the last frame. In VCEG-AZ05, BIO uses a 5x5 window to derive the
    refinement of movement of each sample. Therefore, for one NxN block, the results compensated for movement and The information in gradient corresponding of a block (N + 4) x (N + 4) are needed to derive the refinement in
    sample-based movement for the NxN block. According to VCEG-AZ05, a 6-lead gradient filter and a 6-lead interpellation filter are used to generate the gradient information for the BIO. Therefore, the computational complexity of BIO is much greater than that of traditional bidirectional prediction. In order to further improve the performance of BIO, the following methods are proposed.
    [004] In a conventional HEVC bi-prediction, the predictor is generated using equation (1), where P ( °) and P (1) are the listO and listl predictor, respectively.
    Petition 870190056345, of 06/18/2019, p. 33/90
    3/33
    PConventional Í, j] - Í P ^ [z, j] + P ^ [z, j] + 1 j »1 '(1) [005] In JCTVC-C204 and VECG-AZ05, the BIO predictor is generated using equation (2).
    P Optical Flow = (P (0) [i, j] + P W [i, j] + V x [l, j] (I x { ° } ~ [i, j]) + v ^, j] ( A (0) -A (1) [/, j]) + 1) »1, 2) ^ / · [006] In equation (2), I x <0) and I x (1) represent the gradient directional x in the listO and listl predictor, respectively; I y <0) and Iy (1 ) represent the directional gradient y in the listO and listl predictor, respectively; vx and v y represent the displacements in the x and y directions, respectively. The derivation process of v x and v y is shown below. First, the cost function is defined as diffCost (x, y) to find the best values v x and
    Vy. To find the best values v x and v y to minimize the cost function, diffCost (x, y), a 5x5 window is used. The solutions of v x and v y can be represented using Si, S2, S3, S5 and Sg. diffCostíx, y) x 3P ° (x, y) 9P ° (x, y) i θΡ'ζχ, γ) 9Ρ ! (χ, γ) 2 = £ ( ρ U1) + + v, ---- (P 1 (x, y) - Vx - vy —p 1 , q cLv dy dxdy = Σ (/> (x. v) - P '(. y x> + FFV) + im · ω dx dx dy Dy3 [007] The minimum cost function diffCost (x, y) p O (^ and be derived according to:
    ddijfCost (x, y) = θ ddiffCost (x, y) = θ θνχ '# ([008] Solving equations (3) and (4), v x and v y can be solved according to equation (5):
    S3S5 - SzSe SiSe - S3S2
    Vx = -------------, Vy = ------------ S1S5 - S2S2 S1S5 - S2S2 (5) where,
    Petition 870190056345, of 06/18/2019, p. 34/90
    4/33
    AvL ,, r Av) êPtu Mu ') gP'UvI
    S ^} f .......... „+ .......... (........... '·: ..:' .; ... ..............: .....) (........... ..: .. 2.) Q ex mqc% αχ / y cy
    S, = ... y (£ f / í21 + p « us) ... />'av) i. s 5 = y + tax [009]
    In the equations above the x-direction gradient of a βί pixel at (x corresponds
    y) in figure list 0 corresponds to the x direction of the pixel in
    y) in the figure of the gradient of: 7 ’(x. y) corresponds to list 1 gradient of y direction of a pixel in (x
    y) in the list 0 figure, and corresponds to the gradient of the y direction from a to a to a pixel in (x, y) in the list 1 figure.
    [0010] In VCEG-AZ05, 0 BIO is implemented on top of the HEVC reference software and is always applied to the true bidirectional predicted blocks. In HEVC, an 8-lead interpellation filter for the luma component and a 4-lead interpellation filter for the crema component are used to perform fractional motion compensation. Considering a 5x5 window for a pixel to be processed in an 8x8 CU in BIO, the bandwidth needed in the worst case is increased from (8 + 7) x (8 + 7) x 2 / (8x8) = 7 , 03 for (8 + 7 + 4) x (8 + 7 + 4) x 2 / (8x8) = 11.28 reference pixels per current pixel. In JVET D0042 (A. Alshin, et al., AHG6: On BIO memory bandwidth, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 4th Meeting: Chengdu , CN, October 15-21, 2016, Document: JVETD0042), to reduce the required bandwidth, the necessary data, including pixels offset by
    Petition 870190056345, of 06/18/2019, p. 35/90
    5/33 motion and gradients in the x and y directions will be set to zeroes, if these pixels are outside the current block. As shown in Figure 2, the central square is the original MC region (210) for the current block, and the BIO process requires the predictors and the corresponding gradient in Region A (220) and Region B (230) to derive the offset respectively to further refine predictors using derived displacement and gradients. In JVET D0042, data from Region A (220) and Region B (230) will be set to zero to save the required bandwidth. As the gradient is generated from additional 6-lead filters, the motion and gradient compensated pixels can be generated using the same region as in the original HEVC. Therefore, using the method in JVET D0042, there is no additional bandwidth requirement in the BIO process.
    [0011] However, using additional 6-lead filters to generate gradients in the x and y directions is tricky compared to the original motion compensation design. Two additional 6-lead filters will be required: one is used to generate gradients and the other is used to perform interpellation when the motion vector is fractional. In addition, the block-based process is also proposed to reduce the computational complexity required in VCEG-AZ05. Therefore, a low complexity BIO is presented below. In low complexity BIO, the block-based derivation process is used instead of the pixel-based derivation process, where each block is divided into 4x4 blocks (referred to as BIO blocks in this disclosure) for the BIO-based process
    Petition 870190056345, of 06/18/2019, p. 36/90
    6/33 block. In the original BIO, a 5x5 window is used to derive the movement of each pixel. In block-based BIO, a 6x6 window is used for each 4x4 block to derive motion for the block. Gradient calculation is also simplified by the application of a 3-lead filter with coefficients equal to {-1, 0, 1} in motion compensated pixels according to the low complexity BIO. In Figure 2, the smaller block 250 corresponds to a 4x4 block for the block-based BIO. The block of dashed lines 260 corresponds to the 6x6 window to derive the movement information associated with the 4x4 block. For each pixel within the window, the predictor and gradient must be calculated. For the gradient filter {-1, 0, 1}, the gradient in the x and y directions for pixel 240 in the upper left corner of window 260 needs adjacent pixels shown as black dots in Figure 2. Therefore, the width of required bandwidth is the same as in the original BIO, but no additional 6-lead filter is needed and computation complexity is reduced using the block-based lead process. It is desirable to develop methods to reduce the required memory bandwidth and further improve low complexity BIO encoding performance.
    [0012] In a typical video coding system that uses inter motion compensated prediction, motion information normally transmitted from an encoder side to a decoder, so that the decoder can correctly perform inter motion compensated prediction . In such systems, the movement information will consume some encoded bits. In order to
    Petition 870190056345, of 06/18/2019, p. 37/90
    7/33 To improve coding efficiency, a method of derivation of Motion Vector on the Decoder Side is disclosed in VCEG-AZ07 (Jianle Chen, et al., Further improvements to HMKTA-1.0, ITU - Telecommunications Standardization Sector, Study Group 16 Question 6, Video Coding Experts Group (VCEG), 52nd Meeting: 19-26 June 2015, Warsaw, Poland). According to VCEG-AZ07, the Decoder Side Motion vector derivation method uses two modes of upward conversion frame rate (FRCU). One of the FRCU modes is referred to as bilateral match for B-slice and the other of the FRCU modes is referred to as model match for P-slice or B-slice.
    Pattern-based MV derivation (PMVD) [0013] Figure 3 illustrates an example of bilateral FRCU correspondence mode, in which the movement information for a current block 310 is derived based on two reference figures. The movement information of the
    current block is derived to meet The best correspondence between two blocks (320 and 330) to along the trajectory of 340 movement of current block 310 in two
    different reference figures (ie RefO and Refl). Under the hypothesis of continuous motion trajectory, the MVO motion vectors associated with RefO and MV1 associated with Refl pointing to the two reference blocks 320 and 330 must be proportional to the temporal distances, that is, TD0 and TD1, between the current figure (ie Cur pic) and the two reference figures RefO and Refl.
    [0014] Figure 4 illustrates an example of FRCU model matching mode. Neighboring areas (420a and
    Petition 870190056345, of 06/18/2019, p. 38/90
    8/33
    420b) of the current block 410 in a current figure (ie Cur pic) are used as a model to correspond with a
    corresponding model (430a and 430b) in an picture of reference (that is, RefO at Figure 4). The best correspondence between the model 420a / 420b e the model
    430a / 430b will determine a motion vector derived from decoder 440. While RefO is shown in Figure 4, Refl can also be used as a reference figure.
    [0015] According to VCEG-AZ07, a FRCU_mrg_flag is signaled when the merge_flag or skip_flag is true. If FRCU_mrg_flag is 1, then FRCU_merge_mode is flagged to indicate whether bilateral match merge mode or model match merge mode is selected. If the FRCU_mrg_flag is 0, it means that regular blending mode is used and a blending index is signaled in this case. In video coding, in order to improve the coding efficiency, the motion vector for a block can be predicted using motion vector prediction (MVP), where a list of candidates is generated. A list of merge candidates can be used to code a block in a merge mode. When blending mode is used to code a block, the movement information (for example, motion vector) of the block can be represented by one of the MV candidates in the MV merge list. Therefore, instead of transmitting the block's movement information directly, a merge index is transmitted to the decoder side. The decoder maintains the same merge list and uses the merge index to retrieve the merge candidate as
    Petition 870190056345, of 06/18/2019, p. 39/90
    9/33 signaled by the merge index. Typically, the merge candidate list consists of a small number of candidates and transmitting the merge index is much more efficient than transmitting the movement information. When a block is encoded in a blending mode, the movement information is merged with that of a neighboring block, signaling a merged index instead of being explicitly transmitted. However, the prediction residues are still transmitted. In case the prediction residues are zero or very small, the prediction residues are skipped (that is, the skip mode) and the block is encoded by the skip mode with a merge index to identify the merge MV in the list merge.
    [0016] Although the term FRCU refers to motion vector derivation for upward conversion of frame rate, the underlying techniques are intended for a decoder to derive one or more merge MV candidates without the need to explicitly transmit information from movement. Therefore, FRCU is also referred to as decoder-derived motion information in this description. Since the model matching method is a pattern-based MV derivation technique, the FRCU model matching method is also referred to as the Pattern Based MV Derivation (PMVD) in this disclosure.
    [0017] In the MV derivation method on the decoder side, a new temporal MVP called temporal derived MVP is derived by scanning all MVs in all reference figures. To derive the time-derived MVP LIST_0, for each LIST_0 MV figure in the figures of
    Petition 870190056345, of 06/18/2019, p. 40/90
    10/33 reference LIST_0, the MV is sized to point to the current figure. The 4x4 block pointed by this MV staggered in the current figure is the current target block. The MV is later dimensioned to point to the reference figure that refldx is equal to 0 in LIST_0 for the current target block. The additional dimensioned MV is stored in the LIST_0 MV field for the current target block. Figure 5A and Figure 5B illustrate examples for deriving the time-derived MVPs for LIST_0 and LIST_1, respectively. In Figure 5A and Figure 5B, each small square block corresponds to a 4x4 block. The time derived MVP process scans all MVs in all 4x4 blocks in all reference figures to generate the LIST_0 and LIST_1 MVPs derived from the current figure. For example, in Figure 5A, blocks 510, blocks 512 and blocks 514 correspond to 4x4 blocks of the current figure (Cur. Pic), reference figure LIST_0 with index equal to 0 (that is, refidx = 0) and reference figure LIST_0 with index equal to 1 (that is, refidx = l), respectively. The motion vectors 520 and 530 for two blocks in the reference figure LIST_0 with an index equal to 1 are known. Then, time-derived MVPs 522 and 532 can be derived by scaling motion vectors 520 and 530, respectively. The dimensioned MVP is then assigned to a corresponding block. Likewise, in Figure 5B, blocks 540, blocks 542 and 544 correspond to 4x4 blocks of the current figure (Cur. Pic), reference figure LIST_1 with index equal to 0 (that is, refidx = 0) and reference figure LIST_1 with index equal to 1 (that is, refidx = l), respectively. The motion vectors 550 and 560 for two blocks in the reference figure LIST_1 with
    Petition 870190056345, of 06/18/2019, p. 41/90
    11/33 index equal to 1 are known. Then, the time-derived MVP 552 and 562 can be derived by scaling motion vectors 550 and 560, respectively.
    [0018] For bilateral correspondence blending mode and model correspondence blending mode, two-stage correspondence is applied. The first stage is PU-level matching, and the second stage is sub-PU-level matching. In PU level correspondence, several initial MVs at LIST_0 and LIST_1 are selected respectively. These VMs include MVs from merge candidates (that is, conventional merge candidates, such as those specified in the HEVC standard) and VMs from temporal derived MVPs. Two different starting MV sets are generated for two lists. For each VM in a list, a pair of VMs is generated by the composition of this VM and the mirrored VM that is derived by scaling the VM to the other list. For each pair of MV, two reference blocks are offset using this pair of MV. The sum of absolute differences (SAD) of these two blocks is calculated. The MV pair with the lowest SAD is selected as the best MV pair.
    [0019] After a better MV is derived for a PU, diamond research is performed to refine the pair of MV. The refinement accuracy is 1/8-pel. The refinement search range is restricted to within ± 1 pixel. The final MV pair is the PU-derived MV pair. Diamond search is a fast block matching motion estimation algorithm that is well known in the field of video encoding. Therefore, the details of the diamond search algorithm are not
    Petition 870190056345, of 06/18/2019, p. 42/90
    12/33 repeated here.
    [0020] For the search for the second sub-PU level, the current PU is divided into sub-PUs. The depth (for example, 3) of the sub-PU is signaled in the sequence parameter set (SPS). The minimum size of sub-PU is 4x4. For each sub-PU, several starting MVs at LIST_0 and LIST_1 are selected, which include the MV MV derived from the PU-level, MV zero, TMVP placed HEVC from the current sub-PU and lower right block, MVP derived from the sub -PU current, and MVs left and above PU / sub-PU. Using the similar mechanism as the PU level search, the best MV pair for the sub-PU is determined. Diamond research is carried out to refine the MV pair. Movement compensation for this sub-PU is performed to generate the predictor for this sub-PU.
    [0021] For the model matching merge mode, the reconstructed pixels above 4 rows and to the left of 4 columns are used to form a model. Model matching is performed to find the model best matched with its corresponding MV. Two-stage matching is also applied for model matching. In PU level correspondence, several starting MVs at LIST_0 and LIST_1 are selected respectively. These VMs include MVs from merge candidates (that is, conventional merge candidates, such as those specified in the HEVC standard) and VMs from temporal derived MVPs. Two different starting MV sets are generated for two lists. For each MV in a list, the model's SAD cost with the MV is calculated. The MV with the lowest cost is the
    Petition 870190056345, of 06/18/2019, p. 43/90
    13/33 best MV. The diamond survey is then carried out to refine the MV. The refinement accuracy is 1/8-pel. The refinement search range is restricted within ± 1 pixel. The final VM is the MV derived from the PU level. The MVs at LIST_0 and LIST_1 are generated independently.
    [0022] For the second stage sub-PU level survey, the current PU is divided into sub-PUs. The depth (for example, 3) of the sub-PU is indicated on the
    SPS. The minimum size of sub-PU is 4x4. For each sub-PU on the left or upper PU boundaries, several starting MVs at LIST_0 and LIST_1 are selected, which include the
    MV MV derived from the PU level, MV zero, TMVP placed HEVC from the current sub-PU and the lower right block, MVP derived from the current sub-UP, and MVs from the left and above the PU / sub-UP. Using the similar mechanism as the PU level search, the best MV pair for the sub-PU is determined. Diamond research is carried out to refine the MV pair. Movement compensation for this sub-PU is performed to generate the predictor for this sub-PU. For PUs that are not on the left or upper PU boundaries, the second stage sub-PU level survey is not applied, and the corresponding VMs are defined as the same as the MVs in the first stage.
    [0023] In this decoder MV derivation method, model matching is also used to generate an MVP for Inter mode encoding. When a reference figure is selected, model matching is performed to find a better model in the selected reference figure. Its corresponding MV is the derived MVP. This MVP is inserted in the first position in the
    Petition 870190056345, of 06/18/2019, p. 44/90
    14/33
    AMVP. AMVP represents advanced MV prediction, where a current MV is predictively encoded using a candidate list. The MV difference between the current MV and a selected MV candidate in the candidate list is coded.
    Decoding MV on the decoder side (DMVR) [0024] No JVET-D0029 (Xu Chen, et al., DecoderSide Motion Vector Refinement Based on Bilateral Template Matching, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 4th Meeting: Chengdu, CN, October 15-21, 2016, Document: JVETD0029), Decoder Side Motion Vector Refinement (DMVR) based on bilateral correspondence of models is released. A model is generated using biprediction from the reference blocks (610 and 620) of MV0 and MV1, as shown in Figure 6. Use the model as a new current block and perform the motion estimate to find a better match block (710 and 720 respectively) in Ref. Figure 0 and Ref. Figure 1, respectively, as shown in Figure 7. The refined VMs are MV0 'and MV1'. Then, the refined MVs (MV0 'and MV1') are used to generate a final predicted block predicted for the current block.
    [0025] In DMVR, it uses two-stage search to refine the MVs in the current block. As shown in Figure 8, for a current block, the current MV candidate cost (at a current pixel position indicated by a square symbol 810) is evaluated first. In the first stage search, the entire pixel search is performed around the current pixel location. Eight candidates (nominated by
    Petition 870190056345, of 06/18/2019, p. 45/90
    15/33 eight large circles 820 in Figure 8) are evaluated. The horizontal distance, vertical distance, or both between two adjacent circles or between the square symbol and the adjacent circle is one pixel. The best candidate with the lowest cost is selected as the best MV candidate (for example, candidate at the location indicated by circle 830) in the first stage. In the second stage, a half-pixel square survey is performed around the best MV candidate in the first stage, as shown in eight small circles in Figure 8. The best MV candidate with the lowest cost is the final MV for the final compensation. of the movement.
    [0026] It is also desirable to reduce the bandwidth requirement for the system using PMVD or DMVR.
    BRIEF SUMMARY OF THE INVENTION [0027] Method and apparatus for using standard-based MV derivation (PMVD), bidirectional optical flow (BIO), or MV refinement on the decoder side (DMVR) to refine motion to a predicted bidirectional block are disclosed. According to a method of the present invention, a first motion-compensated reference block associated with the current block is determined in a first reference figure in the reference figure list 0, wherein the first motion-compensated reference block includes first pixels additional adjacent blocks around a corresponding block of the current block in the first reference figure to perform the required interpellation filter for any fractional motion vector of the current block. Also, a second reference block compensated by movement
    Petition 870190056345, of 06/18/2019, p. 46/90
    16/33 associated with the current block in a second reference figure from the reference list of figure 1 is determined, where the second motion-compensated reference block includes additional second adjacent pixels around the corresponding block of the current block in the second figure of reference to perform the required interpellation filter for any fractional motion vector of the current block. The PMVD process, BIO process or DMVR process is applied to generate movement refinement for the current block using reference data corresponding to the first movement-compensated reference block and the second movement-compensated reference block without other reference data outside the first motion-compensated reference block and the second motion-compensated reference block to reduce the required system memory bandwidth. The current block is encoded or decoded based on the motion compensated prediction according to the refinement of the motion.
    [0028] For ο ΒΙΟ, the BIO process may include the calculation of first gradients for first pixels in the first motion-compensated reference block and second gradients for second pixels in the second motion-compensated reference block, deriving the displacement in the x and direction y using the first gradients and the second gradients, and generate the movement refinement based on the displacement in the x and y directions. In addition, the calculation of the first gradients and second gradients can use a 3-lead filter corresponding to {-1, 0, 1} for non-limit pixels of the
    Petition 870190056345, of 06/18/2019, p. 47/90
    17/33 first movement-compensated reference block and the second movement-compensated reference block; and the calculation of the first gradients and second gradients can use a 2-lead filter corresponding to {-1, 1} for boundary pixels of the first motion-compensated reference block and the second motion-compensated reference block to avoid a need for reference data from outside the first motion-compensated reference block and the second motion-compensated reference block; and where the boundary pixels are within the limits of one pixel of the first motion-compensated reference block or the second motion-compensated reference block.
    [0029] In one embodiment, the PMVD process, the BIO process or the DMVR process to generate motion refinement for the current block is skipped to the border pixels of the current block if the PMVD process, the BIO process or the DMVR process for the boundary pixels of the current block require any reference data outside the first motion-compensated reference block and the second motion-compensated reference block. In another embodiment, if the PMVD process, the BIO process or the DMVR process for the current block requires one or more motion-compensated reference pixels outside the first motion-compensated reference block and the second motion-compensated reference block, reference pixels can be generated by filling.
    [0030] A non-transitory, computer-readable medium that stores program instructions that causes a device's processing circuit to execute the
    Petition 870190056345, of 06/18/2019, p. 48/90
    18/33 video encoding method above, is also disclosed.
    [0031] Another method and video encoding device using the bi-directional block-based optical flow (BIO) to refine the movement to a predicted bi-directional block is revealed. According to this method, the current block is divided into one or more BIO blocks based on one or a combination of video resolution of the video sequence, characteristics related to the current block and the current block size. For example, if the contents of the current block are smoother, the current block can be divided into a smaller number of BIO blocks. In another example, if the current block includes a more complicated texture or a more complicated movement region, the current block will be divided into a larger number of BIO blocks. In another example, the current block corresponds to a coding unit (CU) and the current block is divided into one or more BIO blocks according to the size of the current block. In another example, if the current block uses sub-block motion partition, the current block is divided into a larger number of BIO blocks. In another example, if a variation value of the first movement-compensated reference block associated with the current block or a variation value of the second movement-compensated reference block associated with the current block is large, the current block is divided into a larger block number of BIO blocks. In yet another example, if one or more strong edges in the first motion-compensated reference block associated with the current block or in the second motion-compensated reference block associated with the current block is large, the current block is divided into a larger number of blocks ΒΙΟ. The total number
    Petition 870190056345, of 06/18/2019, p. 49/90
    19/33 of BIO blocks for the current block is determined implicitly or flagged explicitly in a bit stream at a sequence level, figure level, slice level, CTU (coding tree unit) or CU level (coding unit ).
    [0032] A non-transitory, computer-readable medium that stores program instructions that cause a device's processing circuit to perform the video encoding method above, dividing the current block into one or more BIO blocks based on the resolution of video of the video sequence, characteristics related to the current block or size of the current block is also disclosed.
    BRIEF DESCRIPTION OF THE DRAWINGS [0033] Figure 1 illustrates an example of Bidirectional Optical Flow (BIO) to derive the displacement motion vector for movement refinement.
    [0034] Figure 2 illustrates the region compensated by movement and its neighboring pixels needed to derive the gradient and displacement related to the Bidirectional Optical Flow (BIO).
    [0035] Figure 3 illustrates an example of motion compensation using the bilateral correspondence technique, in which a current block is predicted by two reference blocks along the movement path.
    [0036] Figure 4 illustrates an example of motion compensation using the model matching technique, where the model of the current block is combined with the reference model in a reference figure.
    [0037] Figure 5A illustrates an example of a process
    Petition 870190056345, of 06/18/2019, p. 50/90
    20/33 derivation of temporal motion vector prediction (MVP) for reference figures LIST_0.
    [0038] Figure 5B illustrates an example of a derivation process of temporal motion vector prediction (MVP) for reference figures LIST_1.
    [0039] Figure 6 illustrates an example of Movement Vector Refinement on the Decoder Side (DMVR), in which a model is generated first using bi-prediction from the reference blocks of MV0 and MV1.
    [0040] Figure 7 illustrates an example of Movement Vector Refinement on the Decoder Side (DMVR) using the model generated in Figure 6 as a new current block and performing the movement estimate to find a block with the best match in Ref. Figure 0 and ref. Figure 1, respectively.
    [0041] Figure 8 illustrates an example of a two-stage survey to refine the MVs of the current block for the Decoder Side Motion Vector Refinement (DMVR).
    [0042] Figure 9 illustrates an example of reference data required by the Decoder Side Motion Vector Refinement (DMVR) for an MxN block with fractional MVs, where a reference block (M + L1) * (N + L -1) is required for motion compensation.
    [0043] Figure 10 illustrates an exemplary flowchart of a video encoding system using Standard-based MV derivation (PMVD), bidirectional optical flow (BIO) or MV refinement on the decoder side (DMVR) to refine the motion for a predicted bidirectional block according to a modality of
    Petition 870190056345, of 06/18/2019, p. 51/90
    21/33 the present invention.
    [0044] Figure 11 illustrates an exemplary flowchart of a video encoding system using bidirectional optical flow (BIO) to refine the movement to a bidirectional prediction block according to an embodiment of the present invention, in which the current block is divided in one or more BIO blocks based on the video resolution of the video sequence, characteristics related to the current block or current block size.
    DETAILED DESCRIPTION OF THE INVENTION [0045] The following description is the best way to carry out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.
    [0046] As mentioned earlier, several motion refinement techniques, such as standard-based MV derivation (PMVD), bidirectional optical flow (BIO) or decoder side motion vector refinement (DMVR) require access to reference data which causes an increase in system bandwidth. In the present invention, techniques for reducing system bandwidth associated with PMVD, BIO and DMVR are disclosed.
    Method 1. BIO bandwidth reduction [0047] In low complexity BIO, Region A (220) and Region B (230), as shown in Figure 1, are used to calculate the gradients, and these gradients are used to derive the shift and refine the final predictor. In order to further reduce the bandwidth, the
    Petition 870190056345, of 06/18/2019, p. 52/90
    22/33 The proposed method according to the present invention does not use data from Region A (220) and Region B (230). To compensate for the unavailability of Region B (230) reference data, the gradient calculation for pixels on the CU boundary is also changed accordingly. In one embodiment, when the 3-lead filter {-1, 0, 1} is used for gradient calculation, a 2-lead filter with filter coefficients equal to {-1, 1} to generate gradients for these pixels at borders ASS. In other words, for a border pixel, the filter {-1, 1} is operated on a current pixel on the border of the current block compensated by movement and a neighboring pixel within the block. In comparison with the 3-lead filter {-1, 0, 1}, the additional pixel outside the block compensated for the current movement that would be needed is not needed now. Since the gradient calculation is different, some normalization may be required based on the filter bypass distance. For example, the gradient calculated using the 3-lead filter {-1, 0, 1} corresponds to the change in pixel values within the pixel distance equal to two, and the gradient calculated when using the 2-lead filter {-1 , 1} corresponds to the change of pixel values within the pixel distance equal to one. Therefore, the gradient calculated when using the 2-lead filter {-1, 1} must be multiplied by 2 to make the change in pixel values within the pixel distance equal to two. In another modality, the BIO refinement process for pixels at CU boundaries is skipped. In another modality, a filling method is used to generate pixel values in Region A (220) or
    Petition 870190056345, of 06/18/2019, p. 53/90
    23/33
    Region B (230) instead of performing motion compensation. For example, the extent of a border pixel directly without mirroring can be used to generate all pixels in Region A (220) or Region B (230). Alternatively, mirror fill can be used to generate all pixels in Region A (220) or Region B (230). After applying filling techniques, the BIO process can be applied without further changes.
    Method 2. Block size in the BIO process [0048] In low-complexity BIO, the block-based derivation process is used instead of the pixel-based derivation process, which can significantly reduce computational complexity. However, using a predefined BIO block size cannot achieve the best encoding gain for videos at different resolutions or with different video content. According to a method of the present invention, the size of the BIO block is adaptive and may depend on the video resolution and / or the characteristics of the video content. For example, for smooth regions, the block size should be larger, to include more textures to obtain an accurate offset. On the other hand, for complicated regions of texture or movement, the block size must be smaller to be adapted to the location. Therefore, in one mode, the block size depends on the video resolution. In another embodiment, the size of the block is dependent on the size of CU. When the CU size is small, the block size must be small. In another mode, the size of the block depends on the use of the partition
    Petition 870190056345, of 06/18/2019, p. 54/90
    24/33 sub-block movement. If the current CU is encoded using sub-block motion partition, a smaller block size will be used in BIO. In another mode, the block size may be dependent on the pixels compensated for the movement. For example, if the range of motion-compensated pixels is large or if there are some strong edges in motion-compensated (MC) regions, a smaller block size will be used for the BIO. The above methods can be combined to implicitly decide on the block size for the BIO process. Alternatively, the block size selected for the BIO process can be explicitly signaled in the bit stream at the sequence level, figure level, slice level, CTU level (coding tree unit) or CU level (coding unit) .
    System bandwidth reduced for PMVD and DMVR [0049] In the decoder-side predictor refinement tools, such as PMVD, BIO and DMVR, the process for refining the predictors generally requires additional reference samples outside the reference block. For example, for an MxN 910 block with fractional MVs, a reference block (M + L-1) * (N + Ll) 925 is required for motion compensation as shown in Figure 9, where L is the derivation length interpellation filter. In HEVC, L is equal to 8. For DMVR search, a pixel area 920 with a pixel width outside the reference block 925 is required for the first stage search within the reference block 925 plus (M + L-1 ) * (N + Ll) plus the 920 ring area. If the best candidate is located on the upper left side instead of the candidate
    Petition 870190056345, of 06/18/2019, p. 55/90
    25/33
    central, additional data outside the ring area 920 can be need users. For example, it is required an area additional L-shaped 930 (ie, a line of pixels additional (M + L-l) and a column of pixels (N + L-1)). Therefore, to support the refinement tools of predictoradditional on the decoder side, width is required. band
    [0050] To reduce the memory bandwidth requirement, a method of filling the filter coefficient is proposed. First, a valid reference block is defined. The valid reference block can be the same as the original reference block (for example, a block (M + Ll) * (N + Ll) 925) or a predefined block that contains the original reference block (for example , the original reference block 925 plus the 920 pixel ring area). Then, when performing MV refinement on the decoder side and / or final motion compensation, any reference pixels outside the valid reference block will not be used. If an interpellation filter needs pixels outside the valid reference block, the filter coefficients are shifted to the boundary coefficient. For example, if a set of filter coefficients with 8 leads is {-1, 4, -11, 40, 40, -11, 4, —1} and the two samples on the left are not in the valid reference block, the filter coefficient set will be changed to {0, 0, -8, 40, 40, -11, 4, -1}. The coefficients of the two samples on the left are rolled over to the third sample (that is, the border sample). In this example, the coefficients for the two samples on the left are {-1, 4} and the coefficient for the border pixel is {
    Petition 870190056345, of 06/18/2019, p. 56/90
    26/33
    11}. The modified coefficient for the border pixel becomes {-1} + {4} + {-11} = {-8}. If all samples in the filter process are in the valid reference block, the filter coefficient set will not be changed. In another modality, if an interpellation filter needs pixels outside the valid reference block, the filter coefficients corresponding to the external pixels will be rolled into the coefficient corresponding to the first valid pixel. For example, the coefficients of two samples on the left are added at the top of the central coefficient, so the center coefficient becomes {-1} + {4} + {40} = {43}. Thus, the filter coefficient set will be changed to {0, 0, -11, 43, 40, -11, 4, -1}.
    [0051] To illustrate an example of this method based on DMVR, the valid reference block in Figure 9 can be the same block as the original reference block (the (M + L-1) * (N + L-l) 925 block). All pixels outside the ring area of a 920 pixel are considered to be invalid reference samples. When performing the filtering process in the search stage or in the final motion compensation, the filter coefficients for the invalid samples will be rolled over to the nearest valid sample. In one embodiment, the coefficient shift is applied only at the research stage, not at the final movement compensation.
    [0052] Different encoding tool may have different configurations of valid reference blocks. For example, for DMVR, the valid block can be the (M + L-l) * (N + L-l) block. For PMVD, the valid block can be the block (M + L-l + O) * (N + L-l + P) where O and P can be 4.
    Petition 870190056345, of 06/18/2019, p. 57/90
    27/33 [0053] In PMVD, a two-stage search is performed. The first stage is research at the PU level. The second stage is research at the sub-PU level. In the proposed method, the valid reference block constraint is applied for the first stage search and the second stage search. The valid reference block for these two stages can be the same.
    [0054] In another implementation method, the pixels needed outside the valid reference block can be generated using some pixels in the valid reference blocks. For example, all pixels outside the valid reference block are filled with the closest pixels within the valid reference block. So, when doing MV refinement on the decoder side and / or final motion compensation, the filled pixels are used instead of the original pixels if the target pixels are outside the valid reference block.
    [0055] The proposed filter coefficient shift or generation of reference pixels can be limited to be applied to certain CUs or PUs. For example, the proposed method can be applied to CU with the CU area greater than 64 or 256, or it can be applied to bi-prediction blocks.
    [0056] THE Figure 10 illustrates a exemplary flowchart in a system in video encoding using derivation in MV based in Standard (PMVD), Flow Bidirectional Optical
    (BIO) or MV refinement on the decoder side (DMVR) to refine the movement to a predicted bidirectional block according to an embodiment of the present invention. The steps shown in the flowchart, as well as
    Petition 870190056345, of 06/18/2019, p. 58/90
    28/33 other flowcharts in this description, can be implemented as program codes executable on one or more processors (for example, one or more CPUs) on the encoder side and / or on the decoder side. The steps shown in the flowchart can also be implemented on a hardware basis, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to this method, the input data associated with a current block in a current figure is received at step 1010, where the current block is encoded using bidirectional prediction. A first motion-compensated reference block associated with the current block in a first reference figure from the figure 0 reference list is determined in step 1020, where the first movement-compensated reference block includes additional first adjacent pixels around a corresponding block of the current block in the first reference figure to perform the required interpellation filter for any fractional motion vector of the current block. A second motion-compensated reference block associated with the current block in a second reference figure from the figure 1 reference list is determined in step 1030, where the second movement-compensated reference block includes second additional adjacent pixels around the corresponding block of the current block in the second reference figure to perform the required interpellation filter for any fractional motion vector of the current block. The PMVD process, BIO process or DMVR process is applied to generate movement refinement for the current block using reference data
    Petition 870190056345, of 06/18/2019, p. 59/90
    29/33 corresponding to the first motion-compensated reference block and the second motion-compensated reference block without other reference data outside the first motion-compensated reference block and the second motion-compensated reference block in step 1040, so that the required system memory bandwidth can be reduced. The current block is encoded or decoded based on the motion-compensated prediction according to the motion refinement in step 1050.
    [0057] Figure 11 illustrates an exemplary flowchart of a video encoding system using bidirectional optical flow (BIO) to refine the movement to a predicted bidirectional block according to an embodiment of the present invention, in which the current block is divided into one or more BIO blocks based on one or a combination of video resolution of the video sequence, characteristics related to the current block and current block size. According to this method, the input data associated with a current block in a current figure is received at step 1110, where the current block is encoded using bidirectional prediction. The current block is divided into one or more BIO blocks based on one or a combination of video resolution of the video sequence, characteristics related to the current block and the current block size in step 1120. A first motion-compensated reference block associated with the current block in a first reference figure from the figure 0 reference list is determined in step 1130, where the first motion-compensated reference block includes first adjacent pixels
    Petition 870190056345, of 06/18/2019, p. 60/90
    Additional 30/33 around a corresponding block of the current block in the first reference figure to perform the required interpellation filter for any fractional motion vector of the current block. A second motion-compensated reference block associated with the current block in a second reference figure from the reference list in figure 1 is determined in step 1140, where the second movement-compensated reference block includes additional second adjacent pixels around the corresponding block of the current block in the second reference figure to perform the required interpellation filter for any fractional motion vector of the current block. A first gradient for each pixel within a first window around a first motion-compensated reference BIO block associated with each BIO block in the current block and a second gradient for each pixel within a second window around a second BIO block motion-compensated reference block each BIO block is derived in step 1150. A displacement vector is generated for each current BIO block based on the first pixel gradients within said first window around said first motion-compensated reference BIO block associated with each BIO block and second gradients for pixels within said second window around said second motion compensated reference BIO block associated with each BIO block in step 1160. The movement refinement is then generated for each pixel in each block BIO based on said displacement vector and the first gradient and second gradient for each pixel in ca of the BIO block in step 1170. The block
    Petition 870190056345, of 06/18/2019, p. 61/90
    Current 31/33 is encoded or decoded based on the motion compensated prediction according to the motion refinement in step 1180.
    [0058] The flowcharts shown above are intended to illustrate an example of video encoding according to the present invention. A person skilled in the art can modify each step, rearrange the steps, divide a step or combine steps to practice the present invention without departing from the spirit of the present invention. In the disclosure, specific syntax and semantics have been used to illustrate examples for implementing modalities of the present invention. A person skilled in the art can practice the present invention by replacing syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.
    [0059] The above description is presented to allow a person skilled in the art to practice the present invention, as envisaged in the context of a particular application and its requirement. Several modifications to the described modalities will be evident to those skilled in the art, and the general principles defined here can be applied to other modalities. Therefore, the present invention is not intended to be limited to the particular modalities shown and described, but must be in accordance with the broader scope consistent with the principles and innovative features described herein. In the detailed description above, several specific details are illustrated in order to provide a complete understanding of the present invention. However, it will be understood by those skilled in the art that the present invention can be
    Petition 870190056345, of 06/18/2019, p. 62/90
    32/33 practiced.
    [0060] The mode of the present invention, as described above, can be implemented on various hardware, software codes or a combination of both. For example, one embodiment of the present invention can be one or more circuitry integrated in a video compression chip or program code integrated in the video compression software to perform the processing described herein. One embodiment of the present invention can also be a program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The invention can also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor or a Field Programmable Port Array (FPGA). These processors can be configured to perform particular tasks according to the invention, executing machine-readable software code or firmware code that defines the particular methods incorporated by the invention. The software code or firmware code can be developed in different programming languages and different formats or styles. The software code can also be compiled for different target platforms. However,
    many different formats in code, styles and languages of codes software and others configuration means code for run at tasks according to the invention
    they will not depart from the spirit and scope of the invention.
    [0061] The invention can be incorporated in other specific forms without departing from its spirit or
    Petition 870190056345, of 06/18/2019, p. 63/90
    33/33 essential characteristics. The examples described should be considered in all respects only as illustrative and not restrictive. The scope of the invention is therefore indicated by the appended claims and not by the preceding description. All changes that fall within the meaning and scope of the claims' equivalence must be within its scope.
    权利要求:
    Claims (6)
    [1]
    1. Video encoding method using standard-based MV derivation (PMVD), bidirectional optical flow (BIO) or MV refinement on the decoder side (DMVR) to refine motion to a predicted bidirectional block, the method characterized by the fact that which comprises:
    receiving input data associated with a current block in a current figure, where the current block is encoded using bidirectional prediction;
    determining a first motion-compensated reference block associated with the current block in a first reference figure from the list of reference figures 0, where the first movement-compensated reference block includes additional first adjacent pixels around a corresponding block the current block in the first reference figure to execute the interpellation filter necessary for any fractional motion vector of the current block;
    determine a second motion-compensated reference block associated with the current block in a second reference figure from the reference list in figure 1, wherein the second motion-compensated reference block includes additional second adjacent pixels around the corresponding block of the current block in the second reference figure to execute the required interpellation filter for any fractional motion vector of the current block;
    apply the PMVD process, BIO process or DMVR process to generate movement refinement for the current block using reference data corresponding to the first
    Petition 870190056345, of 06/18/2019, p. 85/90
    [2]
    2/6 movement-compensated reference block and the second movement-compensated reference block without other reference data outside the first movement-compensated reference block and the second movement-compensated reference block; and encode or decode the current block based on the motion compensated prediction according to the motion refinement.
    2. Method, according to claim 1, characterized by the fact that the BIO process comprises calculating first gradients for first pixels in the first motion-compensated reference block and second gradients for second pixels in the second motion-compensated reference block, deriving the displacement in the x and y directions using the first gradients and the second gradients, and generate movement refinement based on the displacement in the x and y directions.
    [3]
    3. Method, according to claim 2, characterized by the fact that the said calculation of the first gradients and second gradients uses a filter with 3 derivations corresponding to {-1, 0, 1} for non-border pixels of the first block movement-compensated reference block and the second movement-compensated reference block; and said calculation of the first gradients and second gradients avoids the need for reference data outside the first motion-compensated reference block and the second motion-compensated reference block when using a 2-lead filter corresponding to {-1, 1 } for pixels
    Petition 870190056345, of 06/18/2019, p. 86/90
    3/6 border of the first movement-compensated reference block and the second reference-compensated block
    movement; and in which pixels from border are in borders in a pixel of the first block of reference compensated per movement or the second block of reference compensated for movement.4. Method, according to claim 1,
    characterized by the fact that the PMVD process, the BIO process or the DMVR process to generate motion refinement for the current block is jumped to border pixels of the current block if the PMVD process, the BIO process or the DMVR process for the pixels of current block boundary requires any reference data outside the first motion-compensated reference block and the second motion-compensated reference block.
    5. Method according to claim 1, characterized by the fact that if the PMVD process, the BIO process or the DMVR process for the current block requires one or more reference pixels compensated for movement outside the first reference block compensated by motion and the second motion-compensated reference block, one or more motion-compensated reference pixels are generated by padding.
    6. Video encoding apparatus using standard-based MV derivation (PMVD), bidirectional optical flow (BIO) or decoder-side MV refinement (DMVR) to refine motion to a predicted bidirectional block, the video encoding apparatus characterized by the fact that it comprises one or more electronic circuits or processors arranged for:
    Petition 870190056345, of 06/18/2019, p. 87/90
    [4]
    4/6 receiving input data associated with a current block in a current figure, in which the current block is encoded using bidirectional prediction;
    determining a first motion-compensated reference block associated with the current block in a first reference figure from the list of reference figures 0, where the first movement-compensated reference block includes additional first adjacent pixels around a corresponding block the current block in the first reference figure to execute the interpellation filter necessary for any fractional motion vector of the current block;
    determine a second motion-compensated reference block associated with the current block in a second reference figure from the reference list in figure 1, wherein the second motion-compensated reference block includes additional second adjacent pixels around the corresponding block of the current block in the second reference figure to execute the required interpellation filter for any fractional motion vector of the current block;
    apply the PMVD process, BIO process or DMVR process to generate movement refinement for the current block using reference data corresponding to the first movement-compensated reference block and the second movement-compensated reference block without other reference data outside the first block movement-compensated reference block and the second movement-compensated reference block; and encode or decode the current block based on
    Petition 870190056345, of 06/18/2019, p. 88/90
    [5]
    5/6 movement-compensated prediction according to the refinement of the movement.
    7. Non-transient computer-readable medium, characterized by the fact that it stores program instructions, causing a device's processing circuit to execute a video encoding method, and the method comprising:
    receiving input data associated with a current block in a current figure, where the current block is encoded using bidirectional prediction;
    determining a first motion-compensated reference block associated with the current block in a first reference figure from the reference figure list 0, where the first movement-compensated reference block includes additional first adjacent pixels around a corresponding block of the block current in the first reference figure to execute the required interpellation filter for any fractional motion vector of the current block;
    determine a second motion-compensated reference block associated with the current block in a second reference figure from the reference list in figure 1, wherein the second motion-compensated reference block includes additional second adjacent pixels around the corresponding block of the current block in the second reference figure to execute the required interpellation filter for any fractional motion vector of the current block;
    apply the PMVD process, BIO process or DMVR process to generate movement refinement for the current block
    Petition 870190056345, of 06/18/2019, p. 89/90
    [6]
    6/6 using reference data corresponding to the first movement-compensated reference block and the second movement-compensated reference block without other reference data outside the first movement-compensated reference block and the second movement-compensated reference block; and encode or decode the current block based on the motion compensated prediction according to the motion refinement.
    类似技术:
    公开号 | 公开日 | 专利标题
    BR112019012582A2|2019-11-19|video coding motion refinement method and apparatus
    TWI670970B|2019-09-01|Method and apparatus of candidate skipping for predictor refinement in video coding
    BR112019018922A2|2020-04-14|motion refinement method and apparatus based on bi-directional optical flow for video encoding
    CN108028939B|2021-10-15|Method and apparatus for decoder-side motion derivation for video coding
    TWI670966B|2019-09-01|Method and apparatus of adaptive bi-prediction for video coding
    TWI617185B|2018-03-01|Method and apparatus of video coding with affine motion compensation
    BR112019017252A2|2020-04-14|deriving motion vector information in a video decoder
    US20190124332A1|2019-04-25|Inter-prediction mode based image processing method, and apparatus therefor
    BR112019027261A2|2020-07-14|motion vector refinement for multi-reference prediction
    BR112020007329A2|2020-10-06|several enhancements to fruc model matching
    TWI729402B|2021-06-01|Weighted interweaved prediction
    TW201902223A|2019-01-01|Method and apparatus of bi-directional optical flow for overlapped block motion compensation in video coding
    TWI711300B|2020-11-21|Signaling for illumination compensation
    CN112997495A|2021-06-18|Rounding in current picture reference
    EP3788782A1|2021-03-10|Sub-block mv inheritance between color components
    US20190349589A1|2019-11-14|Image processing method based on inter prediction mode, and apparatus therefor
    CN110677674A|2020-01-10|Improvements in interleaved prediction
    CN113302918A|2021-08-24|Weighted prediction in video coding and decoding
    JP2020532896A|2020-11-12|Refinement of the internal subblocks of the coding unit
    CN110677675B|2022-02-08|Method, device and storage medium for efficient affine Merge motion vector derivation
    BR112020003938A2|2020-09-08|motion compensation at a finer precision than the motion vector differential
    KR20180134853A|2018-12-19|Method and electronic device for encoding segments of inter frames
    TWI738292B|2021-09-01|Method and apparatus of simplified affine subblock process for video coding system
    TW202127890A|2021-07-16|Method and apparatus for video conding
    WO2020003260A1|2020-01-02|Boundary enhancement for sub-block
    同族专利:
    公开号 | 公开日
    TWI674794B|2019-10-11|
    EP3556096A1|2019-10-23|
    TW201830968A|2018-08-16|
    CN110115032B|2021-07-20|
    US20190320199A1|2019-10-17|
    CN110115032A|2019-08-09|
    WO2018113658A1|2018-06-28|
    EP3556096A4|2020-05-27|
    US10965955B2|2021-03-30|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20030099294A1|2001-11-27|2003-05-29|Limin Wang|Picture level adaptive frame/field coding for digital video content|
    EP2157799A1|2008-08-18|2010-02-24|Panasonic Corporation|Interpolation filter with local adaptationbased on block edges in the reference frame|
    BR112013031215A2|2011-06-10|2017-06-20|Mediatek Inc|scalable video encoding method and apparatus|
    JP5768662B2|2011-10-31|2015-08-26|富士通株式会社|Moving picture decoding apparatus, moving picture encoding apparatus, moving picture decoding method, moving picture encoding method, moving picture decoding program, and moving picture encoding program|
    CA2923286C|2012-07-02|2019-05-07|Samsung Electronics Co., Ltd.|Method and apparatus for encoding video and method and apparatus for decoding video determining inter-prediction reference picture list depending on block size|
    US11153601B2|2015-11-11|2021-10-19|Samsung Electronics Co., Ltd.|Method and apparatus for decoding video, and method and apparatus for encoding video|
    KR20180129860A|2016-04-25|2018-12-05|엘지전자 주식회사|Inter prediction method and apparatus in video coding system|
    KR20190018624A|2016-05-13|2019-02-25|브이아이디 스케일, 인크.|Generalized Multi-Hypothesis Prediction System and Method for Video Coding|
    WO2017201141A1|2016-05-17|2017-11-23|Arris Enterprises Llc|Template matching for jvet intra prediction|
    JPWO2017204185A1|2016-05-27|2019-03-22|パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ|Encoding device, decoding device, encoding method, and decoding method|
    WO2018084339A1|2016-11-04|2018-05-11|엘지전자|Inter prediction mode-based image processing method and device therefor|EP3435673A4|2016-03-24|2019-12-25|Intellectual Discovery Co., Ltd.|Method and apparatus for encoding/decoding video signal|
    JP6860676B2|2017-01-04|2021-04-21|サムスン エレクトロニクス カンパニー リミテッド|Video decoding method and its device, and video coding method and its device|
    KR20200096917A|2017-12-08|2020-08-14|파나소닉 인텔렉츄얼 프로퍼티 코포레이션 오브 아메리카|Image encoding device, image decoding device, image encoding method and image decoding method|
    EP3804324A1|2018-06-11|2021-04-14|Mediatek Inc.|Method and apparatus of bi-directional optical flow for video coding|
    WO2020013673A1|2018-07-13|2020-01-16|엘지전자 주식회사|Method and device for performing inter-prediction on basis of dmvr|
    CN110809155A|2018-08-04|2020-02-18|北京字节跳动网络技术有限公司|Restriction using updated motion information|
    BR112021002857A2|2018-08-17|2021-05-11|Mediatek Inc.|Bidirectional prediction video processing methods and apparatus in video encoding systems|
    SG11202102569QA|2018-09-13|2021-04-29|Huawei Tech Co Ltd|Apparatus and method for conditional decoder-side motion vector refinement in video coding|
    JP2022506161A|2018-11-05|2022-01-17|北京字節跳動網絡技術有限公司|Interpolation for inter-prediction with refinement|
    WO2020098780A1|2018-11-16|2020-05-22|Beijing Bytedance Network Technology Co., Ltd.|Reference size for inter prediction interpolation|
    CN113170097A|2018-11-20|2021-07-23|北京字节跳动网络技术有限公司|Coding and decoding of video coding and decoding modes|
    WO2020140874A1|2019-01-02|2020-07-09|Huawei Technologies Co., Ltd.|A hardware and software friendly system and method for decoder-side motion vector refinement with decoder-side bi-predictive optical flow based per-pixel correction to bi-predictive motion compensation|
    US11153590B2|2019-01-11|2021-10-19|Tencent America LLC|Method and apparatus for video coding|
    WO2020164580A1|2019-02-14|2020-08-20|Beijing Bytedance Network Technology Co., Ltd.|Size selective application of decoder side refining tools|
    CN113545069A|2019-03-03|2021-10-22|北京字节跳动网络技术有限公司|Motion vector management for decoder-side motion vector refinement|
    WO2020177665A1|2019-03-05|2020-09-10|Mediatek Inc.|Methods and apparatuses of video processing for bi-directional prediction with motion refinement in video coding systems|
    CN112511831A|2019-03-11|2021-03-16|杭州海康威视数字技术股份有限公司|Encoding and decoding method, device and equipment|
    KR20210118188A|2019-03-15|2021-09-29|엘지전자 주식회사|BDOF-based inter prediction method and apparatus|
    WO2020197085A1|2019-03-22|2020-10-01|엘지전자 주식회사|Method and device for inter prediction on basis of bdof|
    EP3963881A1|2019-06-21|2022-03-09|Huawei Technologies Co., Ltd.|Early termination for optical flow refinment|
    CN113596479A|2019-06-21|2021-11-02|杭州海康威视数字技术股份有限公司|Encoding and decoding method, device and equipment|
    WO2021062684A1|2019-09-30|2021-04-08|Huawei Technologies Co., Ltd.|Encoder, decoder and corresponding methods for inter prediction|
    WO2020256601A2|2019-10-03|2020-12-24|Huawei Technologies Co., Ltd.|Method and apparatus of picture-level signaling for bidirectional optical flow and decoder side motion vector refinement|
    WO2021068954A1|2019-10-12|2021-04-15|Beijing Bytedance Network Technology Co., Ltd.|High level syntax for video coding tools|
    WO2021190465A1|2020-03-23|2021-09-30|Beijing Bytedance Network Technology Co., Ltd.|Prediction refinement for affine merge and affine motion vector prediction mode|
    WO2021204234A1|2020-04-09|2021-10-14|Beijing Bytedance Network Technology Co., Ltd.|Deblocking signaling in video coding|
    法律状态:
    2021-10-13| B350| Update of information on the portal [chapter 15.35 patent gazette]|
    优先权:
    申请号 | 申请日 | 专利标题
    US201662437759P| true| 2016-12-22|2016-12-22|
    US201662439200P| true| 2016-12-27|2016-12-27|
    PCT/CN2017/117152|WO2018113658A1|2016-12-22|2017-12-19|Method and apparatus of motion refinement for video coding|
    [返回顶部]